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^ ' We apply a recently developed measure of multiscale complexity to the Gaussian model consisting 

D , of continuous spins with bilinear interactions for a variety of interaction matrix structures. We find 

two universal behaviors of the complexity profile. For systems with variables that are not frustrated, 
an exponential decay of multiscale complexity in the disordered regime shows the presence of small- 
scale fiuctuations, and a logarithmically diverging profile of fixed shape near the critical point 
' ■ describes the spectrum of collective modes. For frustrated variables, oscillations in complexity 

^ indicate the presence of global or local constraints. These observations show that the multiscale 

fH ^ complexity may be a useful tool for interpreting the underlying structure of systems for which pair 

^ . correlations can be measured. 

(D ; 

S ' With the rising interest in complex systems comes a desire to quantify how complex a given system is. Tools that 
> , may be used come from information theory [1-3] and statistical physics [4, 5]. They address questions about how 

^ ' long a complete microscopic description of a system must be, given its constituents and constraints. However, the 
, microscopic information (the entropy) of a system does not correspond well with intuitive concepts of complexity — a 
' system with maximal entropy is merely random, whereas a system with minimal entropy is strongly ordered; systems 

a' commonly considered complex, on the other hand, have rich internal structure, i.e., constraints and correlations. 
This problem may be resolved by the recent approach of considering the complexity on different scales [6-8]: 
I random systems have high information content on small scales; however, microscopic degrees of freedom average out 
(— I , over larger scales, leading to a satisfying description in terms of a small number of variables such as volume, pressure 
Q and temperature. Highly constrained systems, on the other hand, have roughly the same information content on all 
O , length scales. E.g., one can describe a system of many strongly bound particles by the location, orientation, and 
movement of the center of gravity; given the (constant) positions of the particles relative to each other, these few 
^-H [ variables then determine everything there is to know down to microscopic scales. In contrast, truly complex systems 
^ ■ have intermediate levels of organization. For example, a human being has interesting behaviors on a macroscopic 
' level (the length scale of meters), which arises from the level of organs (centimeters), which are composed of cells 
. (micrometers) , which consist of biomolecules (nanometers) . 

A recently developed formalism [7] allows us to calculate the complexity on different levels of observation from the 
underlying probability distribution of the degrees of freedom of the system, which implicitly contains the interactions 
' and constraints of the system. The purpose of this paper is to apply this formalism to Gaussian probability distribu- 
] tions with various structures of the covariance matrix, yielding insight into the workings of the formalism, as well as 
"Y^ • order-disorder transitions in systems with bilinear potentials. 

Ci I The Gaussian distribution is also used in the statistical analysis of biological and social systems [9, 10]. The 
^ I entries of the correlation matrix are often extracted from experimental data. The beauty of Gaussians is that their 
■ probability distribution is completely specified by two-point correlations; however, higher correlations exist. As the 
paper will show, the multiscale formalism offers an opportunity of detecting subsets of variables that form coupled 
functional units, of testing for global or local constraints that manifest themselves in the correlation matrix, and for 
^ i characterizing collective behaviors of systems. 

The paper is organized as follows: Section I reviews properties of correlated Gaussian variables. Sec. II reviews 
the mathematical representation of physical degrees of freedom in the form of Gaussians. Sec. Ill gives an overview 
. of the multiscale complexity formalism and previous results. Section IV then shows how the formalism applies to a 
$H ' variety of different interaction matrices. Section V summarizes and interprets the results. 
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I. GAUSSIAN VARIABLES 



Due to the pervasive power of the central limit theorem that gives rise to Gaussian distributions and due to their 
mathematical convenience, Gaussians are the default assumption for probability distributions under many circum- 
stances. This paper discusses sets of n Gaussian variables, labeled Xi, with i e {1,?t.}. Each has a mean of and a 
variance of af: (xi) — 0, (xf) — erf. The cross-correlations are given by the elements of the covariance matrix R: 
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(xiXj) = Rij. The joint probability distribution is then given by 

To calculate how much information is contained in a Gaussian variable, we use Shannon's information theory [1]; 
however, for convenience, we use natural units (base e) rather than bits; i.e., for one degree of freedom, the definition 
is 

Ix = - j \n{jp{x))p{x)dx. (2) 

for a single Gaussian of variance cr^ this results in = (ln(27r) + l)/2 + In(cr). For the joint distribution in Eq.(l), 
one can obtain 

7x = (n[ln(27r) + 1] + ln(Det R))/2. (3) 

For the trivial case of n uncorrelated Gaussians of variance cr^, one has In(DetR) = 2nlncr, so that the information 
is just n times that of a single variable. 



II. PHYSICAL INTERPRETATION 



There is a close formal analogy between Eq. (1) and the canonical distribution for continuous degrees of freedom 
with bilinear interactions H = JijXiXj: 

where the partition function Z is the integral of the numerator over state space. We will consider two quite distinct 
interpretations of such a system: the first is an individual particle in a harmonic potential with n degrees of freedom, 
with Jii giving the potential along the coordinate axes, and Jij determining the shape of the potential along the 
diagonals. 

The second interpretation considers each degree of freedom as a spin that interacts with other spins. In the context 
of models for magnetic systems, this is called the "Gaussian model" [11] and has been used as an approximation to 
binary Ising spins. In the Gaussian model, it is usually assumed that spins in the absence of interactions follow a 
Gaussian distribution of variance 1 (independent of temperature) , while off-diagonal interactions are weighted with a 
factor of /3, and there is no explicit self-interaction. 

In this paper, for convenience, we consider the self-interaction, which serves to keep spins bounded, as a part of the 
interaction matrix. The relevant control parameter then becomes the ratio of the self-interaction to the interaction 
with other spins, rather than the temperature. We can then write the covariance matrix as R = (— /JJ)"-*^. Changing 
P or changing the magnitude of both self-interaction Ju and off-diagonal interaction Jij only results in a rescaling 
of the variables: (i^XiJijXj = J2^iJij^'j ^i^h = y^Xi. This is a reflection of the equipartition theorem, which 
states that each degree of freedom corresponding to a quadratic term in the energy carries a mean energy of UbT /2. 
As we show in Sec. Ill, such a rescaling only adds an additive term to the microscopic entropy, and does not change 
the terms in the multiscale complexity for scales larger than one. We can therefore set /? = 1 and the self-interaction 
Jii = —1 unless otherwise stated, whereas the interaction with neighbors is proportional to a parameter a which we 
vary to control the system's behavior. 

In contrast to the Ising model, the Gaussian model does not have a well-defined ordered phase. Interpreting the 
system as a particle in a harmonic potential, we find that (a;f) is finite (i.e., the system is in the disordered phase) 
if all eigenvalues of J are positive — otherwise there is at least one degree of freedom with infinite negative energy, 
i.e., a harmonic potential of the form E{x) = —Xx^, A > 0. To model phase transitions more realistically in this 
framework, one would have to introduce quartic terms [12]; however, in this paper we will restrict ourselves to the 
pure Gaussian model. Interestingly, the transition to unbounded spins can happen in systems of any size, and be due 
to the interaction of just two spins. 

The harmonic potential interpretation also offers a geometric argument why the entropy is related to the determinant 
of the covariance matrix (Eq. (3)): the coordinate system can be chosen such that the main axes of the equipotential 
ellipsoid coincide with the coordinate axes (i.e., the interaction matrix is diagonalized) . This is always possible because 
the interaction matrix is symmetric. The quadratic degrees of freedom then decouple; the mean square amplitude 
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along each axis depends on the corresponding eigenvahic of the interaction matrix, and the information (which is a 
measure of the size of state space) is the sum of the information for the individual degrees of freedom. Specifically, it 
is related to the sum of the logarithms of the eigenvalues, or equivalently the logarithm of the determinant. Some of 
the axes of the new coordinate system can be interpreted as aggregate variables such as the magnetization (the sum 
of all spins). 

In the diagonalized form, the susceptibility to external fields at the transition is easy to calculate. In the generic 
case, the smallest eigenvalue Ai can be linearly expanded around 0: Ai = K{a — Qc)- Applying a field h along 
the eigenvector of Ai then shifts the minimum of the energy H (i.e., induces an magnetization) to h/2{K{a — Uc)) 
along this axis. The susceptibility is therefore proportional to (a — Oc) '' with 7=1, independent of the underlying 
dimensionality of the system. Different exponents can arise only if the linear expansion of Ai around has a vanishing 
first derivative, which is not the case for any of the cases we study in the following. 

The link between the entropy (or equivalently, information) and In Det R has one mathematical difficulty: since 
Det R can go to 0, the entropy can diverge to negative infinity. The determinant vanishes if the rank of R is smaller 
than n, i.e., one or more columns can be expressed as linear combinations of others. In our context, this corresponds 
to a variable that is completely specified by a combination of others. Generally, the determinant is a polynomial 
of order n in the coefficients of the matrix, and the order of the zero in question determines how many redundant 
variables there are. The diverging entropy is not a problem for physical systems, since no physical quantity can be 
perfectly specified. 



A suitable measure of multiscale complexity Cn{k) should fulfill several conditions. For the smallest scale, it should 
correspond to the microscopic entropy, which is the information contained in the joint probability distribution of all 
degrees of freedom. If the system is composed of distinct subsets of I variables that are coupled within the subset, but 
not coupled to other subsets, Cn{k) should be for k > I, and take non- vanishing values corresponding to the number 
of degrees of freedom otherwise. Furthermore, the multiscale complexity of a composite of independent subsystems 
should be the sum of subsystem complexities. 

It has been shown in [7] that the following definition uniquely fulfills these conditions: 



Eq. (6) is a sum over all subsets of k variables of the entropy of the system after the subset has been removed. Thus, 
Q{n, 0) is the microscopic entropy, and Q{n, n — 1) is the sum of entropies for individual degrees of freedom. 

It is sometimes convenient to discuss the incremental difference in complexity between scales D{k) = C{k+1) — C{k). 
While C{k) represents the effective number of degrees of freedom of size k or larger, D{k) represents the effective 
number of degrees of freedom at scale k. 

The quantity defined by Eq. (5) has some additional, rather surprising, properties. In particular, it can have 
oscillations and take negative values even for discrete variables. These seemingly anomalous properties have been 
shown to reflect the structure of the system, and speciflcally have been linked to the effect of global constraints on 
local variables, or more generally the concept of strong emergence, in Rcf. [13]. We will explore the relevance of such 
behavior to the Gaussian model. 

To simplify the discussion further, we will show first that rescaling the variables (e.g., by choosing a different inverse 
temperature /3, as described in Sec. II) adds a term to the microscopic entropy C„(l) that only depends on n and the 
scaling ratio, and does not aff'ect higher-order complexities {k > 1). When applied to Gaussians, the terms Q{n,k) 
are sums over the logarithm of determinants of submatrices of R, dropping combinations of k rows and corresponding 
columns. Rescaling the covariance matrix by a factor / yields an additive term of Zln / to the log of the determinant 
of a submatrix of size / x I. Since there are (") such contributions for Q{n,j), the difference between the rescaled 



III. MULTISCALE COMPLEXITY FORMALISM 




(5) 




(6) 




FIG. 1: Multiscale complexities for the 3-spin chain. While all complexities are positive for o > 0, C3(3) is negative for a < 0. 



multiscale complexity Cn{k) and the original is 



,=0 .k-j-ljyj 

= V( i^fc-7--i nljn-j) 
jrC {k-3-l)\{n-k)\3\{n-j)\ ^ 

k — l 

for k > 1 

nln/ for k = 1 



j=0 

The identity in the last line can be derived by expanding (1 + (—1))'^"^ using binomial coefficients. 
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IV. APPLICATION TO SPECIFIC MODELS 

A. Three interacting spins 

Wc start by applying the formalism to a minimal case of 3 interacting spins. The three-spin case captures a number 
of features that will be characteristic of larger spin systems. The complexity profile can be solved analytically giving: 

C'sil) = [3(ln(27r) + l)-21n(l + a)-ln(l-2a)]/2; 
C3(2) = -ln(l + a)-ln(l-2a)/2; 

C3(3) = [-ln(l-2a) + ln(l + a) + 31n(l-a)]/2. (8) 

As Fig. 1 shows, complexities diverge at a = 1/2 and a = —1. These are the ferromagnetic and antiferromagnetic 
transitions to unbounded variables. As this case illustrates, unlike conventional binary spin systems, transitions occur 

even for finite numbers of spins. This result can be understood by recognizing that conventional Gaussian variables 
themselves can be thought of as arising as aggregates of many microscopic bounded variables through the central 
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limit theorem. Negative values of Cs{3) occur for a < 0. This is expected for systems with frustrated spins, due to 
the existence of a constraint on the three spins that does not affect any pair of spins. 



B. Infinite range magnet 



For larger numbers of spins we consider first an infinite-range Gaussian model, with an interaction matrix: 

-1 a ... a 



J = 



a 
a 
a 




(9) 



where a, the interaction between spins, is negative for antiferromagnets and positive for ferromagnets. Inverting the 
negative of this matrix does not alter its structure; one obtains the covariance matrix 



R 




(10) 



with r and c given by 



1 - (n - 2)a 



(l + o)(l-(n-l)o)' 

a 

(1 + a)(l - (n-l)a)' 



(11) 
(12) 



We define the correlation ratio p = c/r, in order to separate the amplitude of the spins from the correlations between 
them: 

(13) 



l-(n-2)a' 

Since ferromagnetic ordering is easier to achieve with infinite-range interactions than antiferromagnetic ordering, 
the transition where all spins collapse into one (p goes to 1) and the amplitude diverges occurs at a small value of 
o = l/(n — 1). Antiferromagnetic ordering occurs at a = —1, where p takes the asymptotic value — l/(n — 1) and all 
variables are maximally anticor related, taking values corresponding to an n — 1-dimensional hypertetrahedron. 
The determinant of R is 



DetR„ = r"(l - p)"^'(l + {n - l)p) 



(14) 



The determinant has a zero of order n — 1 at p = 1, where all variables are the same, and n — 1 of them are redundant. 
It also has a first-order zero at pc = —1 / {n — 1) , where geometrical constraints can be used to eliminate one variable. 

We can now calculate the scale-dependent complexity C„(fc) following Eq. (5). For our example, we can calculate 
this analytically for k = 1, 2, 3 and numerically for other values of k. Since the matrix does not change its structure 
if one or more variables are removed, the determinant has the same form as Eq. (14) with a modified number of 
variables. 

We then have 



Q{n,j) 



n- J 



(ln(27rr) + 1)+ 



- ((n - i - 1) ln(l -p)+ ln(l + (n - j - l)p)) 



which yields 



C„(l) 

Cn(3) 



|(ln(27r) + l + ln(r)) + ^ 



-ln(l-p) + iln(l + (n-l)p); 



[- ln(l - p) - (n - 1) ln(l + (n - l)p) + n ln(l + (n - 2)p)]/2; 

- ln(l - p)/2 + {n- l)(n 2) ln(l + (n - l)p) /4 - 

n{n - 2) ln(l + (n - 2)p)/2 + n{n - 1) ln(l + {n- 3)p)/4; 



(15) 



(16) 
(17) 

(18) 
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FIG. 2: Complexities C„{1) to C„(4) for infinite-range interactions (for n = 15 and amplitude r = 1) following Eqs. (16-18). 
The inset shows Cn(l) for a different scaling of the j/-axis. 



In terms of a, these complexities can be written as 

77 77 — 1 1 

C„(l) = -(ln(27r) + l + ln(l-(n-l)a)) + ^-ln(l-(n-l)a) + -ln(l+a); (19) 

C„(2) = [-ln(l-(n-l)a)-(n-l)ln(l + n)]/2: (20) 
C„(3) = [-21n(l-(n-l)a) + (n-l)(n-2)ln(l + a) + n(n-l)ln(l-a)]/4; (21) 

For n = 3 this reduces to the case of three interacting spins, Eq. (8). All complexities for A; > 1 include a singular 
term — ln(l — p)/2 = — ln(l — (n — l)a)/2 but are independent of r, which is a consequence of the invariance to 
multiplying all spins by the same factor, as described above. C„(l), on the other hand, includes a term ln(27rr) + 1. 

We find two characteristic behaviors for ferromagnetic and antiferromagnetic interactions. In the ferromagnetic 
case (a > 0, p > 0) the multiscale complexity is positive, and decreases monotonically with increasing k. The curves 
of Cn{k) plotted as a function oi k/n approximately collapse to one curve for each value of p > 0, so that the 
complexity can be written as a scaling function C{k/n, p). Near p = 1, this function takes a universal shape, so that 
C{k/n, p) w Ci{k/n) + E{p). The singular term in all complexities pointed out above, — ln(l — p)/2, can be identified 
with E(p). As seen in Fig. 3, reducing 1 — p = pc — p [or equivalently Uc — a, as seen in Eq. (19-21)] by a factor of 
10 increases C„(A:/n) by ln(10)/2. 

The universal shape of C{k/n,p) implies that D{k), the spectrum of excitations, is independent of p near the 
transition. With increasing coherence of the variables, the collective behavior at C(n) increases at the expense of the 
independence of the variables C(l) without affecting the spectrum of intermediate scale excitations. 

In the antiferromagnetic case (a < 0,p < 0) oscillations in Ck{n) arc foimd, as seen in Fig. 4. The amplitude 
of oscillations increases as — /ln(l -|- (n — l)p) — g, where / and g increase rapidly with n and are of order 10^ for 
n = 20. Oscillatory behavior in the multiscale complexity has been linked to global constraints [13]; in this case, it 
is the constraint ^ = which is enforced by the interactions near the transition point, and which can be used to 
eliminate one of the variables. 

It should be pointed out that Eq. (5) is susceptible to numerical inaccuracies, and with insufficient precision one 
observes oscillations for p > 0; however, these are numerical artifacts, as can be seen by increasing the accuracy of 
calculations. The oscillations in the antiferromagnetic regime are not artifacts. 
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FIG. 3: Complexities Cn{k) for n = 40 (squares) and n = 80 (crosses) for the infinite-range ferromagnet, displayed as a function 
of fe/n, for various values of p. Arrows indicate the shift predicted by Eq. (18) 
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FIG. 4: Complexities Cn(fc) for the infinite-range antiferromagnet with n = 15, for different values of p < 0, expressed as 
fractions of pc = — l/(n — 1). This is close to the point where one variable becomes redundant; the multiscale complexity shows 
oscillations of increasing amplitude. 
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FIG. 5: Multiscale complexities for the multi-spin chain, normalized by the number of spins n, for n = 16 (lines), n = 25 
(squares) and n = 32 (crosses) . 



C. 1-D spin chain 



We consider a chain of n spins with nearest- neighbor interactions a and self-interactions —1. The interaction matrix 
is given by 



/ -1 a ... a \ 
a —1 a . . . 

\ a ... a -1 / 



(22) 



For even n, it is not relevant whether interactions are ferromagnetic or antiferromagnctic, since both cases can be 
mapped onto each other by flipping every other spin. For odd n, the asymmetry with respect to a becomes less 
pronounced as n increases: negative complexities are only observed for n < 5, and for n > 15 curves look largely 
symmetric. The following observations assume even n. 

For weak interactions, one flnds correlations that decrease exponentially with distance; spcciflcally, the leading order 
term of Rij is al*"-*'. Due to the short-ranged, one-dimensional interactions, Ising spins with this interaction structure 
do not show an order /disorder phase transition. However, the continuous-spin system displays a transition from finite 
to infinite variances for a = ±1/2, at which point the nearest-neighbor interactions override the self-interaction. 

We find numerically that Cn{k) is proportional to n for small fc > 1: each additional spin takes a finite amount of 
information to describe. This is consistent with the short-range interactions of the model. 

The complexity profile for all scales (Fig. 6) behaves similarly to the infinite range ferromagnet (Fig. 3): as a 
approaches Oc, the complexity profile has a universal shape, C°(/c), that is monotonically decreasing and spans all 
scales. The divergence as a function of a — Oc follows Cn{k, a) = C^{k) — ln(ac — a). 



D. Square lattice 



For a two dimensional example, we arrange spins on a i x L square lattice with periodic boundary conditions and 

nearest-neighbor interactions of strength a. The transition point to unbounded spins is at a = ±1/4. Intuitively, four 
interactions of strength 1/4 are sufficient to balance the self-interaction of strength 1 . 
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FIG. 6: Multiscale complexity for the 1-D spin chain with nearest-neighbor interactions, for different interaction strengths at 
n = 16. 



As with the spin chain we find that the muhiscale complexities are ronghly proportional to the number of spins in 
the system, and are almost symmetric with respect to switching the sign of a. For even L the symmetry is exact since 
a ferromagnetic square lattice can be transformed into an antiferromagnetic one by flipping spins in a checkerboard 
pattern. The symmetry is not exact for odd L and the deviation is largest for small values of L. For different L 
the values of C{k)/N do not collapse as well, indicating a more significant deviation from extensivity than in the 
one-dimensional case. 

The complexity profile shows a similar behavior to the infinite-range ferromagnet: as a approaches cic, a complexity 
profile with universal shape emerges that spans all scales, and is monotonically decreasing, as seen in Fig. 8. A 
logarithmic divergence as a function of a — Uc is found as before. The shape of the complexity profile in the limit is 
different from the previous scenarios; the cases will be compared below. 



E. Block matrix 



We generalize the infinite range Gaussian model by dividing the spins into groups that interact strongly within the 
group, but weakly with members of other groups, yielding the following interaction matrix (we explicitly label the 
self-interaction as —d in this case): 



J = - 



/ d . . . a 

a d a 



a . . . a 
a d a 
a . . . d 



d 

a 
a 



\ 



a 

a 

dj 



(23) 



with m blocks of n spins each. This model is important in the study of complex systems since it has a multi-level 
structure, i.e. modularity, considered to be a universal property of complex systems [6, 14]. 
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FIG. 7: Multiscale complexities for the square lattice with nearest-neighbor interactions, normalized by the number of spins 
n = L X L, for L = 5 (lines), L = 6 (squares) and L = 7 (crosses). 
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FIG. 8: Multiscale complexity for the square lattice with nearest-neighbor interactions, for different interaction strengths at 
L = 4. 
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The determinant of this matrix is 

Det (-J) = {d- a)™("-i) {d+{n- l)a - nb)"'-'^{d +{n- l)a + n{m - 1)6), (24) 

which enables us to find the permissible ranges of d, a, and b: for bounded spins to exist, d, which is the negative self- 
interaction, has to be positive. Positive values of a and b imply antiferromagnetic interactions, negative ferromagnetic. 
The boundaries of the allowed region are given by the zeros of Eq. (24): 

a/d < 1; (25) 
b/d < (1 + in-l)a/d)/n; (26) 
b/d > -il + {n-l)a/d)/{n{m-l)). (27) 

The inverse of this matrix has the same structure as Eq. (23); if we label the coefficients of — as A, B, and D, 
respectively, we obtain 

D = K x[d'^ + {m- 2)(n - 2)nab - (m - l)(n - l)n6^ + 

(n - l)(n - 2)a^ + ((m - 2)nb + (2n - 3)a)d]; 
A = K X {-ad - (n - l)a^ + (m - 2)nab +{m- l)n6^) ; 

B = -Kx {d-a)b, (28) 

where 

K =[{d- a){d +{n- l)a - nb){l + {n- l)a +{m- l)nb)]~^ . (29) 

Since the inverse of the negative interaction matrix is the covariance matrix, this allows us to determine under what 
circumstances correlations to near/ far neighbors are positive or negative. A has a zero for 

bi 2 = — — - — — ((to - 2)na ± Jm^n^a^ + 4(to - \)n{d - a)a \ ; (30) 
2(to — l)n V / 

it is negative between the branches of the root, and positive outside. B is negative for 6 < 0, and positive for 6 > 0. 
Combining the results from Eqs. (25-26) and (30), the phase diagram shown in Fig. 9 emerges. Interestingly, long- 
range interactions do not stabilize the system: at any b ^ 0, the system has a smaller range of stability with respect 
to a than for 6 = 0. 

We now calculate the multiscale complexities. Cnm(l) can be calculated from the determinant Eq. (24) with entries 
D, A, and B from Eq. (28): 

Cnm(l) = ^ {nTO[ln(27r) -Fl] + n(m - 1) ln(i:> - ^) 

+ {m - 1) \n{D + {n- 1)A - nB) + ln{D + (n - 1)A + n{m. - X)B). (31) 

Unfortunately, removing spins does not leave the structure of the covariance matrix unchanged. The determinant of 
the matrix with one spin removed is 

DetC7_i = (D- A)"("-i)-^(D + (n-l)A-nB)""' X 
(Z>^ -I- (2n - ■S)AB -I- (n - l)(n - 2)J^+ 

n{m - 2){D + {n- 2)A)B - n{n - l)(m - 1)B^) . (32) 

Prom this expression, the corresponding complexity for scale 2 can be found: 

C„„(2) = (l/2)[-TOln(D-A) 

+(m - mn - 1) \n{D + {n- l)A - nB) 
-{mn - 1) ln(D + (n - 1)^ + n{m - 1)B) 

+mn\n(D'^ + (2n - 3) AD + (n-l){n- 2)X^ + 

n{m - 2){D +{n- 2)A)B - n{n - 1)(to - l)^^)]. (33) 

The first term in Eq. (33) indicates that the divergence near A = D is logarithmic in mln{D — A)/2; the additional 
factor of TO, compared to previous scenarios, indicates that there are to distinct units. 
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FIG. 9: Phase diagram for block matrix with m groups of n spins each. Relevant variables are the ratios between near/far 
neighbor interactions and self-interaction. Shading indicates the sign of covariances A and B. 

Expressions for higher k do not give additional insight. Wo therefore turn to numerical results, which are shown 
in Fig. 10. Along the b = axis, the system consists of m blocks of n coupled spins each. Complexities C„m(fc) are 
different from zero for A; < n (positive for ferromagnetic interactions, oscillating for antiferromagnetic) , and equal to 
zero for k > n. This reflects the built-in property of the multiscale formalism to identify non-interacting subsets of 
variables. 

For 6^0, complexities at scales up to fc = nm become non-zero. In the quadrant o < 0, 6 < where all 

interactions are ferromagnetic, all complexities are larger than zero. In the other quadrants, one observes either 
negative complexities for higher scales or oscillations similar to those found for the infinite-range antiferromagnet. 



Spin glasses (magnetic systems with random ferro- or antiferromagnetic interactions) are a well-studied model of 
systems with frustrated interactions, multiple degenerate ground states and other interesting features [16, 17]. The 
multiscale complexity of spin glasses is therefore of considerable interest. We consider interaction matrices with 
diagonal elements Ja = —1 and off-diagonal elements = Jji = avij, where is a Gaussian random variable of 
variance 1. To determine transition points for any one realization of the quenched random interactions, we keep the 
set of rij constant and adjust the interaction strength a. 

The first quantity of interest is the critical a at which the transition from bounded to unbounded variances occurs. 
This is related to the eigenvectors of — J, which are necessarily all positive for bounded spins. The spectrum of 
eigenvalues A of Gaussian matrices has been studied [15]; if all entries (including the diagonal) are Gaussians of mean 
and variance a^, the matrix is symmetric, and n is large, it takes the form 



i.e., a semi-circle of a width proportional to ay^. For small n, the cutoff of this semi-circle becomes blurred. 

Calculations show that replacing the random diagonal elements with non-random elements of magnitude d has 
two effects: n is replaced by n — 1 in Eq. (34), and the mean of the distribution is shifted by d (as can be shown 
analytically). Thus, the distribution of A follows P(A) = V4a2(«^^^T^^TA^^rfF/(27rc72(n - 1)). 



F. Infinite-range spin glass 




(34) 
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FIG. 10: Multiscale complexities for block structure interactions with n = 5, m = 3, for different values of a and b. 



The critical interaction value Uc is that for wh ich the s mallest eigenvalue becomes 0, for d = I. Neglecting the 
blurring of boundaries, we thus expect |oc| = rf/i/4(n — 1), which is in good agreement with calculations for large n. 
It should be pointed out that has significant fluctuations for small n — the exact transition point differs for each 
realization of {rij}. These result indicate a significant difference in the behavior of the transition in the case of the 
spin glass and the infinite-range magnet with uniform interactions. The scaling of Uc oc n~^^^ lies between that of the 
infinite-range ferromagnet (a^ cx n^^) and the antiferromagnet (a^ = 1) described in Sec. IVB. 

Using these results, we explore the behavior of the multiscale complexities Cn{j)- Near Oc, we obtain a monotonically 
decaying positive curve, similar to that for the ferromagnetic mean-field magnet. For smaller values of a, the curve is 
shifted to lower values of C„(j), but does not become negative (see Fig. 11). 

Interestingly, the curves of Cn{k) for small k follow similar scaling behavior as those for ferromagnetic nearest- 
neighbor spin chains and lattices: when plotted as a function of a/oc, C„{k)/n roughly collapses onto one curve for 
different n, as shown in Fig. 12. The exact values of the curves depend on the quenched variables, so that averaging 
over different realizations becomes necessary; however, the finite-size effects decrease with increasing n. 

It is significant that the random and partly frustrated interactions do not lead to oscillations in the multiscale 
complexity. Ferromagnetic modes become dominant and trigger the transition before frustration has an impact. 
The study of spin-glasses using replica symmetry breaking has previously found that the spin-glass order parameter 
has a degeneracy of order n in the ground state with spins having a macroscopic ordering in the direction of one 
of these low energy states. Such macroscopic ordering is indeed similar to the ordering found in an infinite range 
model and is qualitatively different from a frustrated infinite range model which constrains the macroscopic state 
to a macroscopically degenerate subspace. Still, we note the different scaling between frustrated and unfrustrated 
interactions described above. 



G. Triangular lattice 

Our final example is a two-dimensional lattice with frustration in the anti-ferromagnetic state, the 2-D triangular 
lattice. Again, we find two transitions points. In the ferromagnetic regime, the transition is a = 1/6 as expected for 
a lattice of coordination number six. The complexity profile at the ferromagnetic transition is qualitatively similar to 
that of the square-lattice ferromagnet, and the multiscale complexity for small k is extensive. In the antiferromagnetic 
regime, the exact point of transition depends somewhat on the extent of the lattice in a;-direction (see Fig. 13): 
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FIG. 11: Complexity profile for long-range spin glass with n = 16, for different values of o/oc, averaged over 100 realizations 
of randomness. 



for example, it is Oc = —1/3 for = 3k, —0.353553 for = 4:k, —0.350373 for = 5k. For that are compound 

minibcrs, the lowest absolute vahic of the transition for any of the divisors is the relevant one. The complexity profile 
near the transition (Fig. 14) now shows slight signs of frustration (small oscillations with k and negative values for 
k fii n), but no large-amplitude oscillations as found for the infinite range antiferromagnet. We interpret this as a 
sign of localized frustrations, consistent with a large ensemble of degenerate ground states, but with only local, rather 
than global, constraints. 



V. SUMMARY 



Comparing the multiscale complexity profiles that were found in the preceding section, some intuitive and some 
rather surprising features emerge. The results are summarized in Table I and Fig. 15. All systems we considered show 
a transition from well-behaved (finite magnitude) spins to diverging amplitude spins when the interaction between 
different spins overrides the self-interaction. 

We find two different universal behaviors of the complexity profile: monotonic decrease with scale, and oscillatory 
behavior. The former is a signature of systems with variables that are not frustrated. At the transition, spins become 
increasingly redundant because they are completely correlated (anticorrelated, in the case of an antiferromagnet) with 
each other. Near the critical point, complexities diverge logarithmically. Fig. 15 shows a comparison between the 
curves near divergence (at 0.999ac, for n = 16) for the interaction structures that show this behavior. The complexity 
profile can be interpreted as the cumulative spectrum of collective behaviors of the system. The independence of 
C{k,p) with changes in p in all these cases implies that D{k) = C{k 4- 1) — C{k) is independent of p near the 
transition. Thus, the the collective behavior at C(n) increases near the transition without changing the spectrum 
of excitations at all scales between 1 and n. One can see that the infinite-range ferromagnet. the square lattice 
and the triangular lattice yield very similar curves, whereas the infinite-range spin glass and the 1-D chain show a 
quantitatively different decay. For the simple ferromagnetic systems (all except the spin glass) the value of C„(/c) for 
large k appears to increase monotonically with spin connectivity. 
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FIG. 12: Multiscalo complexities for the mean-field spin glass, normalized by the number of spins n, for n = 40 (lines), n = 30 
(squares) and n — 20 (crosses), averaged over multiple realizations of randomness. 




FIG. 13: Illustration of the orientation of the triangular lattice. 

Away from the transition, in the weak-coupling regime, one finds that C„(A:) decays exponentially with k. This 
corresponds to fluctuations on small length scales, as one expects from systems in the disordered phase. In systems 
that have local interactions or disordered interactions (essentially, in all the cases we studied except for the mean-field 
ferromagnet and block matrices), multiscale complexities are approximately extensive, i.e., proportional to system 
size, for small k away from the transition. In the language of magnetic systems, this means that one observes a finite 
size correlation length indicating small patches of correlated spins, each of which give a contribution to complexity. 
Finite-size effects (deviations from extensivity) are more pronounced for two-dimensional and infinite-range models 
as compared to the 1-D spin chain, as is to be expected in higher dimensional systems. 

The complexities C„(A;) for k of order 1 thus represent localized fluctuations, whereas nonvanishing Cn{k) for 
k/n of order 1 represent the emergence of collective behaviors on the scale of the system as a whole, and whose 
complexity is therefore not extensive. Since the multiscale complexity is a very general formalism, requiring only the 
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FIG. 14: Complexity profile for the triangular lattice near the antiferromagnetic transition, for different system sizes. One sees 
signs of both frustration and coherent behavior. 



Structure 


Transition: Oc 


Cn{k) extensive 


Oscillations 


three-spin ferromagnet 


1/2 


N/A 


no 


three-spin antiferromagnet 


-1 


N/A 


yes 


inf.-range ferromagnet 


l/(n-l) 


no 


no 


inf.-range antiferromagnet 


-1 


no 


yes 


spin chain, n even 


1/2 


yes 


no 


square lattice, L even 


1/4 


yes 


no 


block matrices 


varies 


varies 


varies 


inf.-range spin glass 


oc 1/Vn 


yes 


no 


triangular ferromagnet 


1/6 


yes 


no 


triangular antiferromagnet 


«-l/3 


no 


weak 



TABLE I: Comparison of qualitative features from from Sections IV A to IV G. 



joint probability distribution for input, it can thus be used to describe fluctuations and to identify phase transitions 
without exphcitly choosing order parameters. 

The second universal behavior, oscihations, is observed when a global constraint leads to redundancy: in the 

simplest case, for the long range antiferromagnet, mutual repulsion enforces the constraint Y^x = 0, implying any 
one spin is determined if all others are known. We find that such constraints do not result from random interactions 
found in spin-glasses, i.e. frustrated interactions do not necessarily result in frustrated variables. There exist enough 
interactions that are not frustrated to ensure collective behaviors of the ferromagnetic type. Oscillations do arise, 
however, for symmetric antiferromagnetic interactions, both between individual spins and between blocks of spins. 

Finally, we note that the characterization that we have provided using the multiscale complexity is distinct from the 
usual characterization of the correlations in spatial systems using a correlation length. The multiscale complexity does 
not require a spatial structure. It identifies the aggregate size of fluctuations in terms of the number of participating 
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FIG. 15: Complexity profiles for difi^crent interactions that show monotonically decreasing C'n(k), at the same ratio a/oc = 0.999, 
for n = 16. The infinite-range ferromagnet, the square lattice and the triangular lattice yield remarkably similar curves. 



spins regardless of the topology of spatial or non-spatial interactions. Thus it provides a more generally applicable 
characterization of the collective behavior in interacting systems. 
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